AView From the Other Side of the Mean

Sensory adaptation and visual aftereffects have long given insight into the neural codes underlying basic dimensions of visual perception. Recently discovered perceptual adaptation effects for complex shapes like faces can offer similar insight into high-level visual representations. In the experiments reported here, we demonstrated first that face adaptation transfers across a substantial change in viewpoint and that this transfer occurs via processes unlikely to be specific to faces. Next, we probed the visual codes underlying face recognition using face morphs that varied selectively in reflectance or shape. Adaptation to these morphs affected the perception of ‘‘opposite’’ faces both from the same viewpoint and from a different viewpoint. These results are consistent with highlevel face representations that pool local shape and reflectance patterns into configurations that specify facial appearance over a range of three-dimensional viewpoints. These findings have implications for computationalmodels of face recognition and for competing neural theories of face and object recognition. How does the brain encode the complex structure of human faces? A complete answer to this question includes both the feature dimensions underlying the neural encoding of faces and the visual information preserved in these codes. Adaptation has been used traditionally as a tool for probing the way basic visual dimensions such as color, motion, and orientation are encoded neurally. Recent studies show that it is possible also to probe the representations of higher-level visual forms, such as faces and complex shapes, using adaptation (Leopold, O’Toole, Vetter, & Blanz, 2001; Moradi, Koch, & Shimojo, 2005; Rhodes, Jeffery, Watson, Clifford, & Nakayama, 2003; Suzuki & Cavanagh, 1998; Watson & Clifford, 2003; Webster & MacLin, 1999; Webster, Kaping, Mizokami, & Duhamel, 2004; Zhao & Chubb, 2001). For example, people identify a face more accurately if they see it immediately after viewing its synthetically created antiface—a face with opposite features (Leopold et al., 2001; Moradi et al., 2005; see Fig. 1). This antiface adaptation is powerful enough to elicit an afterimage of the original face during a brief subsequent presentation of the average face (Leopold et al., 2001; Moradi et al., 2005). Opponent-identity adaptation originates within perceptually salient feature dimensions of faces (Webster & MacLin, 1999; Webster et al., 2004). For example, selective adaptation to the configural elements of face shape (Webster &MacLin, 1999) and to the natural categorical dimensions of gender, ethnicity, and facial expression (Webster et al., 2004) facilitates the perception of faces with opposite values on these features. Opponent adaptation effects suggest that face encodings are based on contrastive neural mechanisms defined relative to average or neutral values of the feature dimensions that represent faces (Leopold et al., 2001). In the present study, we extended the use of adaptation to investigate the nature of the underlying visual representation of facial identity at levels of neural processing that can support view-independent recognition. Previous studies have shown that face adaptation survives two-dimensional affine transformations, such as changes in retinal size and location (Leopold et al., 2001; Rhodes et al., 2003; Watson & Clifford, 2003; Zhao & Address correspondence to Alice O’Toole, School of Behavioral and Brain Sciences, GR4.1, The University of Texas at Dallas, Richardson, TX 75083-0688, e-mail: [email protected]. PSYCHOLOGICAL SCIENCE Volume 17—Number 6 493 Copyright r 2006 Association for Psychological Science Chubb, 2001). The locus of adaptation effects is therefore likely to be in higher-level visual areas beyond those with strict retinotopic organization (Leopold et al., 2001). However, higherlevel visual areas such as inferotemporal cortex or human lateral occipital cortex respond robustly across transformations that involve the three-dimensional structure of objects, such as changes in viewpoint (Booth&Rolls, 1998; Kourtzi, Erb, Grodd, & Bülthoff, 2003; Logothetis, Pauls, & Poggio, 1995; Vuilleumier, Henson, Driver, & Dolan, 2002) and illumination (Hietanen, Perrett, Oram, Benson, & Dittrich, 1992; Vogels & Biederman, 2002). The intermixing of neurons with view-specific and view-independent receptive fields in high-level visual areas (Booth & Rolls, 1998) is consistent with hierarchical models of object recognition. These models achieve view-independent recognition using a hierarchy of neural mechanisms with view-dependent responses (Riesenhuber & Poggio, 1999, 2002). They accomplish this viewpoint-generalization task by learning to associate representations of the two-dimensional layout of features in different views of the same object. The localization of face adaptation effects in higher-level visual areas is not proof that these effects are specific to faces, or that they involve mechanisms responsible for human face expertise. Indeed, robust adaptation effects have been reported for upright and inverted faces when the adapting and test face are in similar orientations—though the relative magnitude of adaptation for upright versus inverted faces varies across studies (Leopold et al., 2001; Rhodes et al., 2004; Webster & MacLin, 1999).Webster andMacLin (1999) concluded that although face adaptation strongly affects the perception of faces, it may not reflect processes specific to face recognition. Rather, they suggested that faces may be particularly useful for studying the effects of adaptation on form perception, because observers are highly sensitive to faces’ configural properties. The demonstrated adaptation to both upright and inverted faces is consistent with an adaptation locus in the human fusiform face area (FFA; Kanwisher, McDermott, & Chun, 1997). Paradoxically, the FFA responds with nearly equal strength to upright and inverted faces (Aguirre, Singh, & D’Esposito, 1999; Haxby et al., 1999; Kanwisher, Tong, & Nakayama, 1998; cf. Haxby, Hoffman, & Gobbini, 2000, for a discussion of these results)—despite evidence that it plays a role in human expertise for faces (e.g., Golby, Gabrieli, Chiao, & Eberhardt, 2001). The effectiveness of different kinds of stimuli for adapting facial-identity perception bears on the ongoing debate concerning whether the underlying representation of faces, presumably centered in the FFA, is modular (Kanwisher et al., 1997) or is based on a distributed network of shared feature codes for face Fig. 1. Illustration of face space (a) and the stimuli used in Experiment 1 (b, c). In the multidimensional vector-space of threedimensional faces, each point encodes the three-dimensional shape and surface reflectance (albedo) of a face. Morphing along the line from the average to an original face covers the range of anticaricatures, as it increases the identity strength. Further increasing identity strength beyond the original creates caricatures. Observers tend to perceive the faces as the same individual, which motivates the concept of an identity trajectory. The antiface of an original scan is located on the other side of the average. The illustrations in (b) show the four original scans that were used in the experiment and their antifaces. The faces in (c) illustrate identity strengths from the experiment, scaled in units of the distance of the original (1.0) from the average (0.0). Using antifaces with identity strengths equal to 0.75 avoided some morphing artifacts that tend to occur at more extreme values. 494 Volume 17—Number 6 Visual Representation of Faces